This application claims benefit of Japanese Application No. Hei 11-304875 filed in Japan on Oct. 27, 1999, the contents of which are incorporated by this reference.
The present invention relates to image processing apparatus, and more particularly to an image processing apparatus having a digital image processing section including enhancement of edges in image to be displayed.
When an image is processed for edge enhancement, a technique is generally used to extract edge components within the image so as to add the extracted edge components to the original image. In such case, the edge components to be added are obtained by extracting high-frequency components of luminance signals of the image. In image pickup devices using RGB primary-color filter, the techniques for generating luminance signal to extract such high-frequency components include: (1) technique for generating luminance signal by using only G (green) signals out of all the color signals of RGB within the image; and (2) technique for generating luminance signal by using all the color signals of RGB in the image.
An example of construction of conventional image processing apparatus will now be explained by way of FIG. 1 in which such two techniques are used to generate luminance signal to perform edge enhancement processing. Referring to FIG. 1, numeral 101 denotes an image pickup means; image signals of Bayer RGB array are obtained from the image pickup means 101. The Bayer RGB array image signals are subjected to white balancing at a white balancing section 102. Luminance data are then generated from 2G or (R+2G+B)/2 at a luminance data generating section 103 where R, G, B represent the outputs of the color signals of R (red), G (green), B (blue), respectively. The luminance data are then passed through a high-pass filter 104 to extract edge components and the degree of edge enhancement is adjusted at an edge enhancement adjusting section 105 to form edge signals.
On the other hand, image signals after the white balancing are separated of colors by pixel as a unit into RGB signals at color separation circuit 106 and then subjected to color correction and xcex3 correction at a color correction/xcex3 correction circuit 107. The RGB signals are converted into Y, Cr, Cb signals at YC conversion section 108. The above described edge signals are then added to the luminance signal Y outputted from YC conversion section 108 at an adder circuit 109 to obtain edge-enhanced luminance signal Y.
The above two techniques have their respective merits and demerits. In particular, FIG. 2A shows a part of image consisting of pixels of Bayer RGB array. In such image of Bayer RGB array, two times of image signals S are used in the case of generating luminance signal Y from 4 pixels of RGB (R+2G+B) as shown in FIG. 2B as compared to the case of generating luminance signal Y as shown in FIG. 2C from two pixels of G. The resulting noise N, on the other hand, is only 21/2 times from the theory of noise reducer. For this reason, supposing S/N at the time of generating from two pixels of G as 1/1=1, that at the time of generating from 4 pixels of RGB becomes 2/21/2=21/2. The case of generating from 4 pixels of RGB becomes 21/2 times better. Since, however, the signals of R, B, which contain not much genuine luminance information, are used as luminance signal when 4 pixels of RGB are used, excessively emphasized edges are generated and defects occur especially at the portion of boundary between two highly saturated colors.
As has been described, there are merits and demerits in both the case of using only G signal and the case of using all the color signals of RGB in generating luminance signal. These are summarized in Table 1.
Prior-art techniques according to some specific documents will now be described. Japanese patent application laid open Hei-10-108208 for example discloses a technique in which contour signals (edge components) are extracted from G signals to perform edge enhancement. Further, Japanese patent laid open application Hei-9-261513 discloses a contour (edge) enhancing method in which two types of contour extracting methods, i.e., MIX (mix) mode and NAM (non-average mix) mode are switched to each other according to the saturation of image. Here, MIX mode refers to generation of contour enhancement signal after addition of a plurality of signals corresponding to the three primary colors of RGB, etc., in the light of image to be displayed; NAM mode refers to generation of contour enhancement signal by adding one selecting the signal emphasized most in the direction of black and one selecting the signal emphasized most in the direction of white out of the respective contour enhancement signals of the above described plurality of signals.
Thus the problem is that there are respective merits and demerits as described above of the technique for generating luminance signal by using only G signals in the image to extract edge components and the technique for generating luminance signal by using all the color signals of RGB in the image to extract edge components. Further, the contour enhancement technique as disclosed in the above Japanese patent laid open application Hei-9-261513 has a problem that, if Bayer RGB array is used as the color filter array of image pickup device, the mere switching between the two types of contour extracting methods according to saturation alone may, depending on the colors of image, fall short of suitable processing due to lack of information. Furthermore, there is a problem of complicated construction in the NAM mode, since it is necessary to perform addition by respectively selecting one emphasized most in the direction of black and one emphasized most in the direction white out of the contour enhancement signals.
To eliminate the above problems in the conventional methods of edge enhancement processing of an image, it is an object of the present invention to provide an image processing apparatus having a digital image processing section including edge enhancement processing of simple construction having only the respective merits of the prior-art techniques.
In accordance with a first aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for obtaining color information of the object from the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; and luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means based on the color information obtained at the color information detection means.
By such construction, an output is provided on the basis of color information obtained at the color information detection means by switching between output of the first luminance data generation means for finding luminance data from G signal and output of the second luminance data generation means for finding luminance data from all color signals of the image signals. Hence edge enhancing components can be extracted on the basis of an optimal luminance data corresponding to the color information, an edge enhancement processing is possible with a favorable S/N and with less edge noise in high saturation regions and low saturation regions. The above object is thereby achieved.
It is another object of the invention to provide an image processing apparatus in which switching of the techniques for generating luminance data for use in edge enhancement processing is smooth and the edge enhancement processing is performed without unnaturalness.
In accordance with a second aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for detecting color information of the object from the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); and means for setting the mixing ratio at the mixed luminance data generation means based on the color information obtained at the color information detection means.
By such construction, mixed luminance data is generated on the basis of color information by mixing the luminance data generated from G signal and the luminance data generated from all color signals with continuously changing the mixing ratio. Hence an optimal luminance data corresponding to the color information can be smoothly changed, making it possible to perform edge enhancement processing without unnaturalness. The above object is thereby achieved.
It is yet another object of the invention to provide an image processing apparatus in which an image processed of edge enhancement at favorable S/N can be obtained even for an image having frequent edge portions.
In accordance with a third aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; and luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means based on the edge information obtained at the edge information detection means.
By such construction, an output is provided by switching between the luminance data generated from G signal and the luminance data generated from all color signals on the basis of edge information obtained at the edge information detection means. Hence it becomes possible to perform edge enhancement processing on the basis of an optimal luminance data corresponding to the edge information even in an image having frequent edge portions. The above object is thereby achieved.
It is a further object of the invention to provide an image pickup apparatus in which, even for an image having frequent edge portions, the techniques for generating luminance data for use in edge enhancement are smoothly switched so that edge enhancement processing without unnaturalness can be performed.
In accordance with a fourth aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); and means for setting the mixing ratio at the mixed luminance data generation means based on the edge information obtained at the edge information detection means.
By such construction, mixed luminance data is generated on the basis of edge information by mixing the luminance data generated from G signal and the luminance data generated from all color signals with continuously changing the mixing ratio. Hence an optimal luminance data corresponding to the edge information can be smoothly changed so that edge enhancement processing without unnaturalness can be performed even in an image having frequent edge portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which an image as processed of edge enhancement at favorable S/N can be obtained corresponding to the characteristic of an object.
In accordance with a fifth aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for detecting color information of the object from the image signals obtained by the image pickup means; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; image region determination means for determining the state of being frequent/infrequent of edge portions (frequency of occurrence per unit area of those regarded as edges, i.e., degree of denseness/sparseness of edges) in the object from the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means; a first selection signal generating section for generating a selection signal for controlling the switching at the luminance data switch means based on color information obtained at the color information detection means; a second selection signal generating section for generating a selection signal for controlling the switching at the luminance data switch means based on edge information obtained at the edge information detection means; and a selection signal switching section for providing an output by switching between the selection signals of the first selection signal generating section and the second selection signal generating section based on a determination signal from the image region determination means.
By such construction, an output is provided by switching between the selection signal for controlling the switching of the luminance data switch means based on color information and the selection signal for controlling the switching of the luminance data switch means based on edge information on the basis of the determination signal from the image region determination means for determining the state of being frequent/infrequent of the edge portions in an object. Hence the edge enhancement processing at favorable S/N can be performed corresponding to the characteristic of the object. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, corresponding to the characteristic of an object, an image is obtained as processed of edge enhancement processing without unnaturalness where S/N is favorable and the techniques for generating luminance data for use in the edge enhancement processing are switched smoothly.
In accordance with a sixth aspect of the invention, there is provided an image processing apparatus having a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for detecting color information of the object from the image signals obtained by the image pickup means; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; image region determination means for determining the state of being frequent/infrequent of edge portions in the object from the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); a first mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the color information obtained at the color information detection means; a second mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the edge information obtained at the edge information detection means; and a mixing ratio setting signal switching section for providing an output by switching between setting signals of the first mixing ratio setting means and the second mixing ratio setting means based on a determination signal from the image region determination means.
By such construction, an output is provided, on the basis of the determination signal from the image region determination means for determining the state of being frequent/infrequent of edge portions of the object, by switching between the setting signal of the first mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the color information and the setting signal of the second mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the edge information. Hence edge enhancement processing without unnaturalness can be performed at favorable S/N and smoothly corresponding the characteristic of the object. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing can be performed without causing unnaturalness at the joint portions.
In accordance with a seventh aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for obtaining color information of the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; and luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means based on the color information obtained at the color information detection means and the joint information obtained from the joint determination means.
By such construction, on the basis of the color information and joint information, an output is provided by switching between the output of the first luminance data generation means for finding luminance data from G signals and the output of the second luminance data generation means for finding luminance data from all color signals. Hence, in joining a plurality of images, edge enhancement processing can be performed without unnaturalness at the joint portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing can be performed smoothly and without unnaturalness at the joint portions.
In accordance with an eighth aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for obtaining color information of the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); and means for setting the mixing ratio at the mixed luminance data generation means based on the color information obtained at the color information detection means and the joint information obtained from the joint determination means.
By such construction, on the basis of the color information and joint information, mixed luminance data is generated by mixing the luminance data generated from G signal and the luminance data generated from all color signals with continuously changing the mixing ratio. Hence, in joining a plurality of images, edge enhancement processing can be performed smoothly and without unnaturalness at the joint portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing without causing unnaturalness at the joint portions can be performed even for an image having frequent edge portions.
In accordance with a ninth aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; edge information detection means for obtaining edge information of the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; and luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means based on the edge information obtained at the edge information detection means and the joint information obtained from the joint determination means.
By such construction, on the basis of the edge information and joint information, an output is provided by switching between the output of the first luminance data generation means for finding luminance data from G signals and the output of the second luminance data generation means for finding luminance data from all color signals. Hence, in joining a plurality of images, edge enhancement processing without unnaturalness at the joint portions can be performed even for an image having frequent edge portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing can be performed smoothly and without causing unnaturalness at the joint portions even for an image having frequent edge portions.
In accordance with a tenth aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; edge information detection means for obtaining edge information of the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); and means for setting the mixing ratio at the mixed luminance data generation means based on the edge information obtained at the edge information detection means and the joint information obtained from the joint determination means.
By such construction, on the basis of the edge information and joint information, mixed luminance data is generated by mixing the luminance data generated from G signal and the luminance data generated from all color signals with continuously changing the mixing ratio. Hence, in joining a plurality of images, edge enhancement processing can be performed smoothly and without unnaturalness at the joint portions even for an image having frequent edge portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing can be performed corresponding to the characteristic of an object at favorable S/N and without unnaturalness at the joint portions.
In accordance with an eleventh aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for obtaining color information of the object from the image signals obtained by the image pickup means; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; image region determination means for determining the state of being frequent/infrequent of edge portions in the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a first luminance data generation means for finding luminance data for use in edge enhancement processing from G signals out of the image signals obtained by the image pickup means; a second luminance data generation means for finding luminance data for use in edge enhancement processing from all color signals of the image signals obtained by the image pickup means; luminance data switch means for providing an output by switching between outputs of the first and second luminance data generation means; a first selection signal generating section for generating a selection signal for controlling the switching at the luminance data switch means based on color information obtained at the color information detection means; a second selection signal generating section for generating a selection signal for controlling the switching at the luminance data switch means based on edge information obtained at the edge information detection means; and a selection signal switching section for providing an output by switching between the selection signals of the first selection signal generating section and of the second selection signal generating section based on the joint information of image from the joint determination means for joint boundary portions of image and based on a determination signal from the image region determination means for those other than the joint boundary portions of image.
By such construction, an output is provided by switching between the selection signal for controlling the switching of the luminance data switch means based on color information and the selection signal for controlling the switching of the luminance data switch means based on edge information on the basis of the joint information of image for joint boundary portions of image and on the basis of the determination signal of the image region determination means for those other than the joint boundary portions of image. Hence, in performing joining processing, the edge enhancement processing can be performed corresponding to the characteristic of the object at favorable S/N and without unnaturalness at the joint portions. The above object is thereby achieved.
It is a further object of the invention to provide an image processing apparatus in which, when joining a plurality of images, edge enhancement processing can be performed corresponding to the characteristic of an object smoothly at favorable S/N and without unnaturalness at the joint portions.
In accordance with a twelfth aspect of the invention, there is provided an image processing apparatus having a joining processing section for joining a plurality of images to generate one piece of image and a processing section for enhancing edges in an image, including: image pickup means for converting an object light into image signals of Bayer RGB array; color information detection means for obtaining color information of the object from the image signals obtained by the image pickup means; edge information detection means for detecting edge information of the object from the image signals obtained by the image pickup means; image region determination means for determining the state of being frequent/infrequent of edge portions in the object from the image signals obtained by the image pickup means; joint determination means for detecting joint information of the image signals obtained by the image pickup means; a mixed luminance data generation means for generating mixed luminance data by mixing luminance data for use in edge enhancement processing generated from G signals out of the image signals obtained by the image pickup means and luminance data for use in edge enhancement processing generated from all color signals of the image signals with continuously changing the mixing ratio (including 1:0 and 0:1); a first mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the color information obtained at the color information detection means; a second mixing ratio setting means for setting the mixing ratio at the mixed luminance data generation means based on the edge information obtained at the edge information detection means; and a mixing ratio setting signal switching section for providing an output by switching between setting signals of the first mixing ratio setting means and of the second mixing ratio setting means based on the joint information of image signals from the joint determination means for joint boundary portions of image and based on a determination signal from the image region determination means for those other than the joint boundary portions of image.
By such construction, an output is provided as switched between the setting signal of the first mixing ratio setting means for setting the mixing ratio of the mixed luminance data generation means based on color information and the setting signal of the second mixing ratio setting means for setting the mixing ratio of the mixed luminance data generation means based on edge information on the basis of the joint information of image for joint boundary portions of image and on the basis of the determination signal of the image region determination means for those other than the joint boundary portions of image. Hence, in joining a plurality of images, the edge enhancement processing can be performed corresponding to the characteristic of the object smoothly at favorable S/N and without unnaturalness at the joint portions. The above object is thereby achieved.